Description:

Add opt-in support for the experimental batch sender (open-telemetry/opentelemetry-collector#8122). When opting into this functionality the exporter's Consume* methods will make synchronous bulk requests to Elasticsearch, without additional batching/buffering in the exporter.

By default the exporter continues to use its own buffering, which supports thresholds for time, number of documents, and size of encoded documents in bytes. The batch sender does not currently support a bytes-based threshold, and is experimental, hence why we are not yet making it the default for the Elasticsearch exporter.

This PR is based on #32632, but made to be non-breaking.

Link to tracking Issue:

#32377

Testing:

Added unit and integration tests.

Manually tested with Elasticsearch, using the following config to highlight that client metadata can now flow through all the way:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
        include_metadata: true

exporters:
  elasticsearch:
    endpoint: "http://localhost:9200"
    auth:
      authenticator: headers_setter
    batcher:
      enabled: false

extensions:
  headers_setter:
    headers:
      - action: insert
        key: Authorization
        from_context: authorization

service:
  extensions: [headers_setter]
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [elasticsearch]

I have Elasticsearch running locally, with an "admin" user with the password "changeme". Sending OTLP/HTTP to the collector with telemetrygen traces --otlp-http --otlp-insecure http://localhost:4318 --otlp-header "Authorization=\"Basic YWRtaW46Y2hhbmdlbWU=\"", I observe the following:

• Without the batcher config, the exporter fails to index data into Elasticsearch due to an auth error. That's because the exporter is buffering and dropping the context with client metadata, so there's no Authorization header attached to the requests going out.
• With batcher: {enabled: true}, same behaviour as above. Unlike the batch processor, the batch sender does not maintain client metadata.
• With batcher: {enabled: false}, the exporter successfully indexes data into Elasticsearch.

Documentation:

Updated the README.